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ABSTRACT 

This report synthesilzes the information presented at 
three conferences designed to develop guidelines for future funding 
for Follow Through programs . 'These conferences were part of a series 
of activities undertaken by the National Institute of Education (NIE) 
in 1981 to plan for a ijew set of follow Through research and pilot 
^project aet-ivities. After the foreword in chapter I, chapter' II 
discusses the concept of Follow Through from 1967 to 1972. Chapter 
III focuses on planning for Follow Through, reporting NIE's 
suggestion that systematic change be brought by having pilot projects 
^oous on implementing and managing proven procedures rather than on 
developing new. curri|;ula or. instructional practices. Also discussed 
in this cDi^ter are four crucial problems in the management of 
coitfpensator|( education:, (1) improving the instructional process; (2) 
building internal support systems; (3) utilizing external support 
groups; and (4) seeking selective evali^ati^n. Chapter IV describes 
evaluation str^egies for Follow Throi^h and provides discussion of 
the value of conducting a "multiple case, study** to measure program 
effectiveness, instrumentation, effectiveness of studying one system, 
and the value of conducting a ifulti-site study. Concluding 
observations are made in Chapter v. A list of 29 background papers 
prepared for NIE are included -in Chapter VI. (AS) 
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I . FOREWORD 



This report synthesizes a series of activities under- 
taken by the National Institute of Education in 1981 to plan 
for a new set of follow Through research and pilot project . 
activities. 

Under an agreement signed 'in, June^KiSO by ^e Office of 
Elementary and Secondary Eduation and the Office of Educa- 
tional Research and Improvement, NIE wds directed to use a 
iimall portion (4 tb 6 percent) of the Follow ihrbugh budget to 
fund research that cpuld benefit not only Follow Through, but 
other compensatory education initiatives as well. NIE will, 
in collaboration with Follow Through, encourage the develop- 
ment, ^estiivi^, and dissemination of improved oon^nsatory edu- 
cation j^rograms infused with re search -based knowledge. Rather 
than develop additional curricular or learning models, the NIE 
activities will build upon available knowledge about what 
makes education effective. 

In August 1980, $400,000 of Fiscal Year 1980 funds were 
^ansferred from Follow Through to NIE so that the Institute 
dould coimmence to plan wa^s in which, to obtain the best think- 
ing possible about the direction of new Follow Through RandD. 

initiated a variety of activities. Including coramissioning 
^papetrs; tolding three invitational 'conferences, and conducting 
two hearings at which representatives of the public expressed 



their views about the future of Follow Through. The confer- 
ences and hearings all occurred in February 1981, timed to co- 
incide with the development of procurement guidelines for 
1981-82. " ' ' 

The first of the conferences, hosted by the Portland- 
(Oregon) Public Schools, and the second, hosted by the Schoo> 
District, of Philadelphia, focused on programatic issues. The 
thir^ conference, hosted by the Austin -(Tfexas) Independent 
School District, focused on docximentation of the approaches. 
Many experts who had b^fR commissioned by NIE to write papers 
on Follow Through attended these conferences and shared their 
findings. The 100-plus conference participants included per- 
sons with prior experience with«FolloW Through and similar 
programs, representatives of constituencies served by Follow 
•Through, and individuals with broad interests in the field of 
compensators' education. Participants were jointly selected by 
NIE and the Follow Through program in collaboration with the 
host school districts. 

The authors of this report, Beatrice Gross and Ronald 
Gross, were commissioned in 1976 by the 0. .S. Office of Educa- 
tion to Write one of the American Education Bicentennial « 
Essays "jl Nation of Learners." They axe co-authors of several \ 
^ texts in the field including Will It Grow In A Classroom? , 
The Childrens* Rights Movement , and Radical School Reforrn . 
Future Directions fdr Open LeaTrning by Mr. Gross was published 
bv the Institute in I9B0, and his. The Arts and the Poor was 
published by the U. S. Office of Education in 1968. 



II .» wAat 'is follow through ? ' 



•The Follow Through proferaai began in 1967 with the broad . operational 
goal of Baking ^choolfhg nore^ effective for lov-i<ncoB^ children .and 
ilBproving their life chances by building .upon t^e gains these chil- 
dren had made in Head Start. The intent was to naintain those gains 
by an «itension of educational intervention and ccaaprehensiv^ serv- 
ices through the th^d grade. The program was to be a ftajor service - 
progran, as extensm as dead Start, hut focusing on older children. 
Funds for Follow Through were provided in the legislation for th^ war 
on poverty through the Office of Economic Opportunity (OEO), but OEO 
transferred the funds to the U. S. Office of Education (USOE) which 
had agreed to administer the program. 

•- 

**In 1968, it became obvious that the nationwide service program 
initially envisioned could not be carried out, due to the small 
allocation for Follow Through made by the Office of ^onomic Opportu- 
nity. The cqipbasis was shifted, therefore, to a focus on research 
and development with the maintenance of service in fewer coonunitles 
a subsidiary goal. Follow Through *s research and development effort 
ims to )>e directed toward determining what kinds of alternative edu- 
cational models could make important differences in the achievement 
of young,! economically disadvantaged children. The research method- 
ology adopted was called planned variation. A unique concept called 
model-spohsorship was developed, and the variety of educational al- 
ternatives eeneratild by model^spoxlBors with different philosophical 

Excerpted from: Follow Throughts Forces for Change jn the Primary 

Schools . S) The Hifth /Scope Press, 1980 



and theoretical orientations vas to coiistltute the planned variations. 
Evaluation mod comparison of the results of #iodel-sponsor *8 efforts 
were to be conducted by the USOE and ij:s independent contractdrs. 

"The" variations teong the educational models inclxided one using a 
token economy, another stressing direct instruction and drill, a third 
employing a cognitive frtimevork, and so on. Models based on the use 
of open classrooms, concepts of developmental Interaction, partner- 
ship between home and school, . parent education, T)arent supported di- 
agnostic tactics, bilingual curricula. Interdependent learning games, 
cul^urai-lingulstic and cultural-democratic approaches, mathemagenlc 
activities, non-graded organization, and responsive environments were 
tested, as well as some combinations ot these different strategies. 
Therory, research, and practice were cqmbined in an unique effort to 
empirically test hov each of these models, as Implemented in local 
schools, affected the learniifg of economically disadvantaged chil- 
dren. . , % , ' y_ 

"At its- peak in the early, 1970s, the Follow Through program In- 
cluded 22 model-sponsors! , 173 local-projects, 30 state education 
agencies, 84,000 children «id their parents, more t,han 4,000 teachers, 
teachii^ assistants, and home visitors and hundreds of 'school princi- 
pals, building supervisors, and other administrators. In addition, 
hundreds of parents, along with representatives of* a variety of commu--' 
.nity agencies, served on policy advisory committees in support of this 
national program* ^ ' - ' 

/■ 

•'The various participants in Follov Through shaped the way in which 
any one of the given educational models was implemented at a gl^en 
site. The experience with this process ofc assimilation and acc»jpmoda- 
tlon has become one of the significant contributions of Follow Through. 
Variation in both implementation and ef f^ctivehfss from site-to-^ite Is 
a jcheme that underlines the difficulties inherent^ in the educational 
change process. " 

•'In spite of many difficulties, the majority of^ these model- 
sponsor /loc«l-^project relationships still exist. ^ The Follow Through 
program is one of thg largest, longest, hnd 150s t often s.tudied educa- 
tional intervention efforts ever conducted under the 'auspices of the 
federal govenment. flhe Elementary^i/d Secondary Education Act — 
Title I program is mammoth compared to Follow Through, but that pro^ 
gram was designed as a service" Iprogr am and not as an experiment in 
•education.,) y ' ^ ^ 

/'As early as 1970, concerns regarding the national evaluations 
were voiced by federal officials, model-sponsors, and local-project 
personnel. The Department of Health, Education and Welfare^ the Of- 
fice of Management and Bbdgc^t, the Office of Education, and branches 
within the Office of Education Were unable to agree wholly on vhlch 
Follow Through outcomes should be evaluated. .Concern about a program 
phase-out initially scheduliid to begin the 1974-1975 achool year led 
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■any Follow Through parents and others to make strong stands vigor- 
ously attesting to the worth of follow Through. As a result, in 1973. 
the isOE reversed its earlier phase-out decision and continjied to fund 
entering classes of children each fall. 

"Gradually, decisions were made concerning the evaluation plans 
for Follow Through. The focus of the evaluation .was narrowed consid- 
erably as child outcome measures were reduced in number and model- 
sponsors with too few local-projects were eliminated nation- 
al evaluation (though they continued to function and. In fact, have 
provided some sponsor-specif Ic evaluation data.) 
t 

"Beginning in 1972 the Follow Through program became a year-by- 
ycar program. Both model-sponsors and local-S>rojects were led to 
believe that each .year was the last. These uncertainties contributed 
to Instability of both model-sponsor and Jocal-project staffs and 
made pUnniag for research an4 development less systematic than it 
might have bfeen with concerted long-range planning.. 



Current!ly, the future of Follow Through, • like that of all 
federal soc4.al programs, is uncertain. Specific questions are 
being raised by the legislative branch regarding whether Fol- 
low Through, in .particular, is sufficiently unique or successT 
. f ul to merit continued funding. 
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III. PLANNING FOR FOLLOW THROUGH: THE FIELD RES PONDS 

■ — \ ? : ' — ' 



Faced vith this challenge of platming for a 
research and develops^nt prograsi,' the NIE, -^ten 
years ago, would like,Xy have done all the plan- 
ning itself, froB Washington, with a fev con- 
sultants. But ve*ve learned to li«ten to the 
field. That's why we have cc^ out into the 
field to. join with practitioners, theorists, 
■and researchers to device the best strategy. 
We'jre here to listen, and*to learn. 

Charles Stalford, MI£ 
Opening Remarks to 
Portland Conference 



A. rOLLOW THROUGH; A TRADITION OF CONFLICT 

Can Follow Through generate enough conpensvts on key ii 
sues to survive? It atust , according to Portland's opening 
speaker, Douglas Carnine, director of the University of 
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KIE FOLLOW THROUGH FLANKING CONFERENCE 

PORTLAKD, OREGON " * . FHILADELPHU, PESNSYLVAKIA 

February 4-5, 1981 February 10-11, 1981 



•X 



Day One 

8:00 a.m. Registraticm » 

Local lntroductlonp.«n* Welcome ^ 
In|roduction to PjesAnt Follow Through Prpgraa 



8:30 
9:00 
9:15 



A 



9:45 
10:00 
10:15 



thrcrview of KIE Fallow Thro^igh Acilvliies, and Conference 
Purpose and Processes ** ^ 

J)iscu6si9n of^ Systemic Change, and Charge to Small Groups 
BREAK 



Convene in Siaall Croups to: Introduce Yourselves, Dis- 
cuss and Reach Consensus on the Syst^silc Change Concept, 
'and Revise the NIE Statenen^ of Systemic Change 

11:30 LUNCH ' 

1:00 p.m. Reconvene in General Session for Reports from Small Groups 
and Discussion of Systemic Change 



1:5,0 
2:00 



:30 
4:45 



Instructions for Small Group Sessions 

Reconvene in Small Groups fc^r: Srief SuaDarles of Com- 
missioned Papers,. Discussion of Responses to the KIE ^ 
Questionf, and Generation and Discussion of Additional 
Questions 

Reconvene in General Session for Progress. Review 
ADJOURN • * 



Day Tvo 
8:45 

10:45 . 
11:15 

11:45 



Reconvene in Small Groups to: Answer Any Remaining 
Question^, Pjioritize Suggestions, aiftdiDiscuss Report to 
General Session ' /. 

BREAK (Group leaders arid reporters prepare report.) 

Reconvene in General Session for Reports frcsn Small 
Groups . , . 

Discuss Reports and Issues 



12:30 p.m. General Session Adjourns 
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Oregon's Direct Iv>strufction Follow Through model. But consid- 
ering its history of conflict, and the_>einper of the times, 
this will not be easy. . , ^ ~ . ■ ^ 

During the late 1960's, with the War on Poverty and Great 
Society programs, educators had high hopes that l^hey could 
find sure answers to the question of how to educate the dis- 
advantaged. By 1970, ovex* 20 program development groups ' 
(called sponsors) representing the entire gamut of educational 
philosophies were participating in the Follow Through-program.. 
Each sponsoring institution belieyed it had a methodology, that 
would work.' With government officials coordinating the ef- 
forts, sponsors sharing their expertise with skilled educa- 
tors* and involved parents helping to motivate students, 
everyone expected that Follow Through students would surpass 
the students outside the program and that. the program would be 
an unquestioned success. Although problems in t^ evaluation 
procedure made it difficult to determine the success of Follow 
Through, and disagreements still exist over whether the na- . •. 
tional evaluation accurately reflects the results of the pro-^ , 
gram, most people agree that the program has not produced the 
kinds of dramatic changj that had been anticipated. 

* . 

"Conflicts arose over ' resource allocations, over program 
materials, over time allocations for the program, over goals 
of education," recalled Carnine, who believes that members of 
the Follow Through community roust now focus t^heir research on I 
certain crucial questions. Why do some sites fail to imple- 
ment the models while others succeed? Are some sponsors more 
successful at transmitting thei^T models and,- if so, why? What 
is a reasonable compromise between fidelity to a prescribed 
model and a local adaptation? Can local innovations be effec- 
tively institutionalized? Must Follow Through prq^rams in- 
volve the forinal school organization,, or can th^y. bypass them? 
Can sponsors be faded out with some essence of the innovation 
left intact? 

"Follow Through bas a potential for great success and 
great failure," concluded Carnine. "Success >rill come from 
selutions addressing consensually based needs. Follow Through 
must generate at least a limited consensus on certain key is- 
sues, for without some resolutions 1 don't believe that Follow 
Through will survive." 

The very question of how successful Follow Through has 
bs itself part of the conflict. After a brief recital of 
Rstory of Follow Through, Patricia Olmstead, director of 
(diversity of North Carolina- Follow Through program who 
id the.l>hiladelphia conference, pointed out deficiencies 
in the national evaluation. Components like parent mvoiviB- 
ment and its effect on children's social and psychological^ 
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^ ^ well-being* Were slighted in favor ]bf a narrow focus on chil- 
dren's academici aphievement. 



• 



«. • 1^ SYSTEMIC C^NGE THE ANSWER^ ' 

Educators have lefarhefl ,a great deal from the last fif- 
teen years* experience^ with compensatory education programs, 
despite the many still-uAanswered questions^ NIE suggests 
that to*capitalize on this hard -won knowledge, the pilot 
projects focus on implementing and managing proven proce- 
dures , rather than on developing new curricula or instruc- 
tional practiced. ' 

^o>^licit .feedback £rom the field on this strategy, NIE 
presented the ^following statement to the conferees at Fort- 
land and Philadelphia and asked for ' reactions and sugges- 
tions: 




KIE believes that the first wave of new. pilot projects should not' 
focus upon new curricula or instructional practices per se. Basic 
•research and earlier cxpetieaces in Follow Through prograas indl- , 
cate that there are 'Many instructional practices that can be effec- 
tive, if aanaged oy inpleaented properly > ^hc first new Follov 
Through approaches will therefore focus oiT deronstratii^ sew ways* 
in which lea's (Local" Educational Agencies) cJn overicoae barriers 
to effective instructional nanagraent and lnpleacntation. 

Illustrative thenes around which pilot projects fbr^tbe New Strand 
i approache| sight be organized to use such knowledge^ include: 

• < l^teans to increase instructional tlse in Follow Through 

classrooms through isproved managc^nt services; . 

e New patterns of staff develppaent and selection of staff , 
-^o gain better instructional 'sanagenent; including co- » 
operative agreements between schools, teather education 
^ institutions and teacher associations or unions; ' 

• . New ways to syste^tically involve parent and coosminity 

'groups in planning and conduct of Follov Through prograns, 
including the use of patents and families to provide 
instruction Jin the Iwmm or conunity; 

t * ' ■* 

• Mew uses of inforaation syateas, including asscssaent and 

' • evaluation results, to bring better diagnostic and prescrip- 
tive inforoMtion to bear on Follqv Thi^ough student learning 
needs; 
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. • New ways to facilitate support of school building and 

district. adninlstrators for substantial changes typically 
' * required by innovative Follow Through procedures. 

VIE believes that the new pilot projects should focus on systemic 
change. That is, they should not deal simpl]^ with chyiges In the 
^classroom or bther single xomponcnt. Rather, they should be de- 
signed to brdng about changes in the whole sy&ten required to 
• deliver instruction and should result in increased coordination 
and support within* that system. 

Docuaentation and evaluation of the pilot projects will be of major 
importance. NI? eacpects to provide sui^>ort- to each pilot project to 
assist tbem in tlie design and execution of a documentation and eval- 
uation system. The system should help the projfect identify weak- 
nesses so that these may be corrected and provide information that 
external audiences interested in losing all or part of the project 
can use to determine its effectiveness. Of pArticaUr interest in 
the documentation and evaluation system will be methods of deter- 
. mining how well the project is implemented. 

» - 

• Sy&temicvchange might be a fruitful focus for the next 
generation of Follow Through progi^ains, but, according to the 
participants at Portland and Philadelphia, the concept must be 
limited by certain constraints, controlled by certain values. 

Some were fearful that what mide Follow Through distinc- 
tive — the comprehensive service component — would be lost. 
Others were clearly repelled by the idea of a top-do%m, sys- 
tem-wide mandate. Not one of the vrork groups accepted the 
rubric of systemic change with enthusiasm. Since each group 
member had only to look across the table to find someone^who. 
held an oppiosite educational philosophy, it was easy to iTOa- , 
gii?e that any systemrwide mandate could outlaw or impede an 
individual's own preferred teaching style. 

If systemic change was' to be the theme of future Follow 
Through research, participants at both conferences demanded 
assurances that staff and parents, wpuld be involved m plan- 
ning new mbdels and that programs! would be assessed roQre accu- 
rately than in the past: 

• "No curriculum model or systemic change could be 
adopted, even if it promises speedier basic skills 
acquisition, which excludes parents or disregards the 
importance of teacher commitment or downplays chxi- 
dren's physical, social, emotional, and cultural 
needs . * . • , 
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0 • "Staff consensus is- the befet assurance that the pro- 
gram would be implemented . " 

• "The learning ani? cultural style of the local popula- 
tion should be considered^** ^ 

• "This time, require entry data, better assessement 
and more accurate evaluations.** . 

The case for popular acceptance was roost dramatically 
articulated after the first day's meetings. "Say we knew 
children drilled in multiplication tables 10 hours a d^ 
would be brilliant math students within the year," said one 
Portland participant. "What could we do with that knowledge 
if we also knew from experience that: one, the kids would 
resist coming to school;. two> teachers would quit because 
their jobs were boring and, the hours wer'e too long? three, the 
unions would back the teachers; four, the school system 
couldn't afford i^> and five, thf community f felt that in- 
creased reading and social skills were priority goals for 

farly childhood." Her example was the most, concrete formula- 
ion of a frustration expressed by practitioners who wanted ^ 
the next orant to be at least as attractive vto children, par- 
ents, tea^eirs, apd school systems as it is to researchers and 
developers. . * ^ 

Taking exception to this -position was an articulate mi- 
nority of research-oriented behaviorists at b<>th conferences. 
They asserted that a radical change in the schools is heeded 
and the kind of a change is'obvious to anyone who looks at the 
Foyiow Throu^ evaluations produced 4)y Abt, etc. rWe have 
proven that dramatically higher basic skills cttJ^t^ncy is 
produced with the direct instructional model," said, a leading 
bbhaviorist% "If we set up a coordinated model inicluding the. 
organization of instruction to increase time-on-task,\ and if 
ve select and train teachers who use our test^ and pxoven 
teaching methods faithfully, we can guarantee success. a. Wfe^ 
have found the way to eliminate dysfiinfctional teaching styles. 
The method is teicher-proof . The kids all learn td read^and 
compute." < 

Philosophical differences emerged in all work groups. \ 
In one Portland group.it took several hours of sparring for 
discussants tb find a common ground; they agreed to urge nie 
to fund a research project that would "yield valid results, 
nie group suggested a new, controlled five-stage research 
study. 

STAGE Is The search for exemplary programs "that help 
children leatn better than anyone could reasonably ex- 
pect" whjch could be inodels for funding. This *»t)Uld 



include particularly successful existing programs and un- 
tried models which Follow Through experts believe' have 
the elements necessary for success. 

STAGE 2: A detailed description of the successful pro^ 
grams; all of the factors which might help explain their 
success, as well as detailed descriptions of the schools, 
the districts, and communities (i.e., political, social, 
economic factors) in which they exist. 

STAGE 3: Development of a verifiable theory of effec- 
tive early primary education (EEPE) including^ necessary 
adaptations to student differences and other system 
variables. 

STAGE 4: Implementation, and replication of alternative 
models of JEEPE. i . 

STAGE 5: A study of the factors that make for effective 
replication. . ♦ • 

"Without a welj-de-f ined accurate descriptioh of the model 
we don't know what , happened or why it happened,* began Walter 
Hathaway, the evalua^bn specialist from the Portland Public 
Schools. "Without 4. i^ok at the context of where it happened 
and how it happened W don't know what went into making it 
happen. V-.,,^^ 

"We must ''know the 'where, how and what* to know if it can 
be traiisf erred from one site to another or if it can be {'gen- 
eralized at all. So after we understand the model, we h£^ye to 
take a look at 'receptivity* of the receiving system (the'man- 
agement, the staff, students, parents,' etc.)." Walter Hatha- 
way called it a "holistic model," or systemic change, and -dia- 
grammed it for the assembled'conferees. This group's sugges- 
tion that the "ecology" of the successful site be thoroughly ; 
described was elaborated upon by the Austin • conferees. The 
consensu^ by those most concurring with the transportability 
of a successful program, was that if detailed descriptions of 
implementation included "climate^ of the scl^l, the school 
district, and the community, replication els^p^here would be 
predictable. 

*^ Most east coast educators at the PhiJ!adelphia conference 
advocated building on the existing Follow Through system. No 
%#ork group in Philadelphia developed a "start over from 
scratch scheme" comparable to the five-stage one just de- 
scribed. In fact, they were skeptical of any such effort, as 
indicated by ET5 based Marianne Amerel,\who conmiented, "There 
seemed to be near consensus that the best way to waste the 
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available funds would be Jnount several new pilot projects 
that would take two years to 'rev up,' and would be ready to 
function just in time to be cut off froin further "supiwrt. • 
This will teach us once again that, it takes at least two years ' 
to rev up a project." ' - / 

The four work groups meeting in Philadelphia cited five 
priorities higher than systemic change: 

1. Better Coordination of Federal and State Programs is 
a priority management issue, especially how, when 
money. is in short supply, 

2 , Research on Perpetuating, Maintaining, and Replica - 
ting a Program In a state of high excitement and 
productivity is equally important. Following a study 
of why certain programs work over a period of time, 

♦ research should focus on how to replicate that re- 
sult in other systems. . ^ \ 

3, "Uniform School Learnings" should take precedence 
over systemic change or time- on -task. Although the 
federal governmerit cannot mandate "common intentional 
school learnings,** a concept described in Jdhn 
Porter's paper, school systems^ applying for grants 
could be required to come up with such a list. ^ A . 
well-thought-out compendium of expecta^tions which 
clarifies what is expected of teachers would enable 
them to plan their class time more intelligently. 

4. Parental Involvement and Comprehensive Services are 
essential to the integrity of the program;. Although^ 

. participants did not expect NIEto fund them, they 
did hope some funding could be arranged. 

It could be argjjted that some of the priorities listed ar^ 
examples- of systemic change. Definition was a probleo with 
which each group grappled: how many systems within a system 
-had to be affected for change to be considered systeaic? One 
giroup drew the following diagram to show that many systems ex- 
isted simultaneously, any of which could be targeted for 
rhanje. 
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C. i ^rOWARDS BETTER MANAGEMENT OF c6MPEWSATORX^gl3UCATION 

Any (effort to improve schooling for disadvantaged chil- 
dren must address four crucial problems; these were explored 
by work groups at the Portland and Philadelphia conferences: 

1. Improving the, JEtistructional Process 

♦2. Buildirm-.'^ternal Support Systems 

^ 3. Ut>arxzing External Support Groups 

Seeking.' Selective Evaluation \ 




\ 

1 , Improving the Instructional Process 

Three pillars support the achieveroent of good results in 
compensatory education, the conferees affirmed: good, well- 
trained teachers; a "team" approach that involves teachers, 
parents, administrators, and curriculum resource persons; an 
eclectic pedagogy that respects the need of children to work 
an<^ succeed at medaingful classroom tasks. . 

Recru it Committed Teachers . "Court the teachers. Tell them 
what kind of assistance they will get. Tell them what kind of 
staff training will be involved and how much time it will 
take. Tell them what new skills they must master and if tho^e 
skills will serve them after the project is over . . . Treat 
them with respect . . . Get them to volunteer for the^ new 
program.** These were among the provocative guidelines sug- 
gested by Robert Stahl, of the California Teachers' Associa- 
tion. "Unless an enthusiastic explatnation of what th6 project 
can dp for*-a teacher is given, expect reluctance in implement- 
ing Follow Through or any other innovation," he cautioned. 

After selecting the best teachers and before beginning 
inservic6 training, programs should develop methods to main- 
tain teachers* commitment to Fo.llow Through, the conferees 
advised. 

• 'Give participants "a piece of the action" byjinvolv- ^ 
ing them in planning the inservice program. / 

• Find ways to get teachers' honest reactions to the 
training as it proceeds. 

• Consult with participants about scheduling staff 
development programs and be prepared to offer training 
during the instructional day or to pay teachers for 
their overtime. 
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\ Invite professional ^llegial interactions among par- 
ticipants. 

• Focus on what teachers do, not yhat they are . 



• Build the program around whatsis already vrorking ef- 
fectively in the school. 

• Make the, staff development program responsive to a 

J school needs-assessment in which the teachers parti- 
cipate. * ( 

• Provide a plainly written explicit description q€ the 
program goals and practices. 

* * 

Staff training iAself should be planned carefully. - After 
selecting teachers who are committed, treat them with* respect 
and consider theii needs, urged the work -grctup. * 

• Keep meetings short and distribute a written agenda in 
advance. 

• Include both theory and practical ideas for the class- 
room. 

■ • Spend some time on the technology of the instructional 
model and the 'management of class time. 

' . • ■ 

• Keep the expectations and capabilities of the staff in 
mind when training them. 

. • Bring in an evaluator to help define the project, to 
work with the teachers to decide on the scope and size 
of the experiment, and to design the formative and 
siimmative evaluations. 

• Give teachers the tools, the time, and the training to 
enable them to observe and diagnose children and make 
informed decisions -abolit their learning needs. 

• Train substitute teachers along with the regular 
staff. 

• Recognize that programs will vary^from site to site. 

• Advise teachers how they can get help 'from outside 
resources such as Teacher Education Associations, 
Teacher Resourcf, Centers, Networks for Teachers, 
Teacher Exchanges, and Trained Wolunteers. 

'■ 

• '■ • ■*• V ' 

U - 
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♦ Increase Engaged Time . Conference participants in Portland 
and Philadelphia insisted on a broad interpretation of the tiroe- 
on-task focus of the ''sta^tement provided to them by the NIE. 

"Time-on- task equal's the'^nthutes per day the child is 
worifeng at a high success rate," stated one grpup. "Time is 
not the product — student achievement is.' We should be meas- 
uring the quality of the time and the results it produces," 
concluded another group.- • r 

Arguing against a narrow def initior^ of time-on-task, 
William Spady, dir^tctt- of the American Association of School* 
Administrators* National Center, for the Improvement of Learn- 
ing, admonished, "Look at the individual child and meet the 
needs of that child. Timing — not time-on-task — may be the 
most important factor for achievement. If the task is ap- 
propriate J . . then the time spent willibe fruitful, engaged 
time. That's wh&t we should measure." 

Both th^ Portland and Philadelphia groups agreed that in- 
creasing engaged time was urgent, and they suggested ways it 
'could be accopiplished. , ^ 

• ' * . 

• Change the teacher's »ole to get students working 
independently and intef gating collaboratively. 

• Refine tlfie definition of -'•task" to apply to clear, 
' reasonable, and attainable goals. 

• Assess the performance of students and teachers in 
relation to the achievement of the tasks. 

# ' ■ "■ • 

• Install a management information syst^ which helps 
the teacher to keep track of the child's skill level 
and indicates the next appropriate task. 

• , Reduce interruptions of learning time that occur when 

children or teachers are required to leave class. 

• Match instructional strategies to learning objectives 
and children's learning styles. 

Provide varied approaches, adequate practice time, 
and multiple opportoinities for learning and success. 

^ ' • .Bnphasize that children come to class with cultural 

and personal histories that must be taken into" consi- 
deration when designing an appropriate curriculum. 

• Refer staff to NIE publication Tiife To Learn by 
Carolyn Denham and Ann Lieberman, • published by the 
U.S. Government Printing Office.^ 
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involvfe Parents. Follow Through programs would do well to 
consider what kind of parent involvesnent is ^fsxrable aiid to 
plan- how to achieve this. All the partiexpaats-who-^cused on 
parental contributions were enthusiastic about this component 
' of Follow Through, and several at the Philadelphia meeting 
were personally knowledgeable about the successful collabora- 
tion betwedKschooi personnel and trained iJarents that has 
been documented in such places rfS^Richmond, Virginia; Jack- 
sonville, Florida; and the state of North Carolina. ' 

In contrast, the Portland discussants suggested under- 
taking a study to "prove" the effects of parent participation 
on their children, if only to Validate "what everyone feels. 

As a result of a thoughtful task-analysis, discussants in 
Portland and Philadelphia listed six distinctive kipds of par- 
ent involvement. I 

• PoUtical goal setting, including lobbying for 
national funding . ^ ' 

• Setting the direction for actual programs and col- 
laborating with staff in making decisions including 
budgets, proposals, staff training, and curriculum. , 

• Participating in the delivery process, by serving 
asMnstructional aides, and relieving teadhers of 

^ clerical tasks. 

• Supporting the school program by helping^ children 
study at home. ■ ^ 

• Learning, as students themselves. 

• TrainimAn skills such as decision-making, so they 
can JrtSdel the kinds of behavior that increase one s 
effectiveness. * , 

2. Building Internal Support. Systems 

Teachers need Assistance and positive reinforcement from 
on-site resource people, from administrators who are trained 
to* understand the program and its gpals, from P??"^^J;° 
spect their work, and from the parents of the children with 
i^m they are working. The Portland and Philadelphia work 
groups that focused on Internal Support Systens suggested ways 
to ensure that teachers receive such support. 
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Admini strative Support . Conference participants emphasized 
the crucial role of principals. They must b^open, sensitive 
people t BhJfB to lead and yet share responsibility, profession- 
als who understand compensatory education programs, early 
development, and the importance ^of a supportive educational 
climate. In shifting staff for the Follow Through program, 
the principals must consider the welfare of the children, the 
teachers and- the prpgram as a whole. 

' ' * • " 

If principals are to Cope with additional responsibili- 
ties, they may need relief from some chores, and recognition 
for th^ir new efforts. Other certified persons may be able to 
share some of the day-to-day duties like budgeting and *schea-^ 
uling, and district administrators should agree to evaluate, 
principals not only on management skills but on educational 
leadership. ^ . • ^ 



Peer SupF>ort . Both for the morale of the teachers and admin- 
xstrators engaged in the prograio, and in ordeit to encourage < 
syste||{-wide change, staff in the Follow Through programs need 
support from their colleagues throughout the local system, 
argued the work-groups. In some schools. Follow Thirough 
teacl||rs have been assigned to separate dining rooms. To 
avoici jealousy and misunderstanding that can lead to this kind 
of iiplation, it is essential to disseminate information on 
Follow Through goals and programs throughout the local system. 



Parent Support .' Confinning the opinions of tKeir colleagues 
who participated in the work-groups on Improving the Instruc- 
tional Process, members of the Internal Support work-groups 
emphasized the impqrtance of having parents collaborate with 
teadhers and administrators to develop and implement plans for 
the Follow Through progreuns . Tl;^y advocated the use of par- 
y ent*-in active, productive roles and stressed the desirability 
of j^roviding training to parents. . 

Commilnity Support . Pubricity can aid teachers and aidministra- 
tors by providing volunteer assistance and positive reinforce- 
ment from the fcoromuirity, but only- if the. information is trans- 
mitted in a lively, understandable foVm. Churches, neighbor- ^ 
hood associations, and local radio and TV stations ar6 willing 
links to ihe community, declared the Portland work-group. 

The Philadelphia group proposed an Innovative alternative 
to the Portland suggestion of sending out neys releases. Com- 
munities could study' their owtt programs (surveying the commu- 
nity pupil data, etc.), tturning dissemination xfito an active 
process, and by directly engaging in the research, heighten^g 
their interest in the results. -a *t 
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Guidelines for Judging Grant Applications . Asked to suggest 
KFP guidelines regarding the Instructional Process and Intei;- 
nal. Support Systems,- the work-groups proposed that applicants 
address: / ^ 

a. The attitudes and ex|^ctat,ions of the sta.ff, with 
special attention to the involvement of the princi- , 
pal. / . > . . 

b. 'the organizatibnal structure and procedures including 
' -student/staff •assignments; tijne allotments, ard tole 

responsibilities. - 

c. The system of incentives and rewards that will be 
used to encourage Ihigh level performance. 

• d. If and how teacher associations wiJl be included in 
the negotiations. » 

e. If and how the local Board's commitment to the new 
model has )^een assessed. 

f . How the applicant plans to introduce new materials., 
new $tructures, new teacher behavior, and new 
values. 
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g. How the applicaiTt plans to work with the staff so 
. "* that" new structures, behavior, and values are "in- 
' ternalized" by the staff. 

The work-groups further advised that proposals inclVide 
provisions for: 

• Sufficient financial support to cover the cost of the 
innovation. • 

• Explpring multiple funding sources which would guaran- 
tee that the project could run for two three years. 

• A plan that is responsive to a demonstrated local 
need. * ^ ' 

' A well-thought-out delivery system whitfh includes the 
training of parentis. 

• An effective mamagement system. 

A. 

• • A plan for involving sponsors or outside- advisors who 
can help maintain the quality X)f. the curriculum, -train 
fitaff , and work with teachers and parepts without 
causing conflict. * ^ 
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• A staff member to collect .data on program i^lementa- 
' tion, so that what is learned from this experimental 

program can be used by others who might want to r§pli- 
cate it. ' » 

3. Utilizing Ext-ernal Support Groups 

In one of the iew instances of divergent outcomes from 
Portland and Philadelphia, two very different reports emanated 
from the work-grbups discussing External Support Grcups. 
Philadelphia's report focused oh ."Involving External Support 
Groups," while Portland* s. report on the role of ispoiisors could 
be approprf&tely^ t^tled'^Phasing Out External Support Groups." 

Involving External Support Groups (Philadelphia) . Schools and 
school systems need more resources, and they need to learn howy 
to use what they have more efficiently. Schools are not al- 
ways able to ta)ce advantage of the myriad federal, state, and 
local programs designed to help them provide ser^ces to chil- 
dren. Sponsors can,pffer services and resources directly and ' 
can link schools with outside resources. 

Another key role for sponsors should be to publicize Fal- 
low Through 's discoveries. If their students and supervising 
teachers know about the program, teacher -training .institutions . 
with students in the field can sow the Foiyow Through seeds in 
other public school systems with which theXr come iato contact. 

Jobn W. TM^ter, president. Eastern Mibhigan University, 
suggested that public schools might supplement paid staff with 
undergraduate volunteers as %^11 as students getting credit 
from teacher-training institutions. Youngsters w!io require 
more individual attention than they can get in a staii^ard 
school day profit from the arrangement. 

Members of the Philadelphia group emphasized the recip- 
rocal relationship between the university sponsor and the pub- 
lic school. While sponsors can provide schools with services 
and, expertise that the ,^4d1s themselves lack, spwisors learn 
the>t^hniqu6s and inforroatjion they must transmit in pre- 
service education from the schools they work with. 



Phasing Vut Externa l Support Groups (Portland) . low xs the 
time to begin the proc ess of phasing oiit the techaical assist- 
ance that sponsors have been providing, before the funds are 
cut, suggested the Portland group. By making «ie»elves avail 
able on an "on call" basis, rather than routinely doing^Mg^, 
sponsors can help shift responsibilities to local sites^pp 
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The Portland work-group considered the implications of 
their recommended phase-out of technical assist^nqpe. Rural 
schools that have counted cmi their sponsor's liaison with 
state and federal agencies will probably have a particularly 
difficult' time learning to go it alone. £f^onsors anticipating 
this problem should help these systems to fill the gap. As 
Follow Through programs a ttemp t^g j^ become §elf -sufficient, 
cost factors will have to be d^^Mere&Jfn making the choice 
of an educational model. Qn" ""BlfPus side, increased cor 
ordination with other federal programs within 4 school sys- ' 
tem, which has always been advocated, will probably.be sought 
in a systematic way, how that programs find they must co- 
ordinate their efforts to balancje their budgets. . 

4. Seeking Selective Evaluation 

Follow Through evaluation studies have been characterized 
as limited in- value because of the flawed program design and 
the restricted range of nfeasures. The Philadelphia work-group 
on, Evaluation considered what components arei desirable in fu- . 
ture evaluation procedures and agreed on six priorities* 

• » ' ' ' 

a. "Long-term effects should be evaluated,^ stated 
Robert Egbert, who was the first national- director of 
Follow Through. Children should be studied over a 
ten-year period, with special emphasis on social be- 
havior as in the Lazar research on long-term effects 
of pre-school education. , "Such a study would include 
the number of students repeating grades, those- as- 
signed to Special Education, those who engaged in 
acts of delinquency, etc. Lohg-term measures on Fol- 
low Through graduates should alsc> include traditional 
achievement tests, measures of progress towards for- 
mal operations, measures of conceptual level, etc. 

In view of the extreme variability of rfesults from 
project to project, institutional case studies should 
be made of a few carefully selected successful and 
unsuccessful projects." 

b. Evaluation should go hand in hand with program defi- 
ni;tion and development to avoid repeating the flaws 
in the national Follow Through evaluation. "There 
was little match between' our methods and theories and 
the Stanford Research Institute (SRI) test battery," 
pointed out Lorraine Smithberg, director of the Bank 
Street-Follow Through program, in her paper. •'"We 
would have welcomed a study of the incoming children 
that would have told us more about their language 
level, motivation and maturational needs.** Smithberg 
also expressed a need for large-scale documentation 
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at each site that would de-scriljlP the starting point 
of staff, children, and parents; the interactive dy- 
namics; th^ behavior of . the power structure? etc. >^ 
This data would make all change detectable; growth 
of any aspect could be monitored; and mechanirsms that 
produce growth, identified . ^ . * 

Documentation of- programs should,- indicate what a pro- 
gram is in reality, not just in intent,, and bow- it 
survives replicailon, ' "There has never been any ma- 
jor, sepai;ately identified funding for a study of the^ 
degree of model implementation at the 'classroom and 
project level," declare^, Egbert . 

Studies of consumer satisfaction should be conducted, 
and an ongoing record of sponsor/site co^unieTations 
aAd relationships maintained: 

* 

Parent involvement should be researched. 

« • 

All major aspects of the supportive services aeed 
assessment. 
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VChat Evaluati on Data is Useful to the Field? Does collect^ing 
massive amo unts^ data— on incoming students, the progress 
and implementatio?Kof the program, the health of the children, 
children's in-school and at-home behavior— put an unusual bur- 
den on the school system and on the teachers? Does collecting 
such data perpetuate an unreal sponsor/federal world that has 
nothing to do with school reality? Should principals protect - 
teachers from this kind of evaluation effort -so they can be 
free te^oncentrate their best efforts on teaching? 

The work-group on. Evaluation examined these questions and 
concluded that the field should not accede to unqiitestioned 
evaluation designs simply to satisfy governynent personnel. ' . 
Rather, it should insist on^ the kind bf 3"^"^^^^ J^^^J^^^^J^^^" 
tative data collection that can -be directly, useful . to teachers, 
as well as to sponsor^ and to -i^ederal officia^ls. All projects 
do not have to have the same tests— they merely need to use 
- ones similar enough to permit comparisons. ^^^^^P^* "'^^^L.j^e 
effort should be made to'tievelop -testing systems" that revolve 
around matrix-sampling, Rasch (latent trait) Item Banks, and 
case studies. , . 
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IV. EVAHJATION STRATEGIES FOR FQU^OW THROUGH 



To aaxiaiize the usefulness of lAiat is l«artied 
frcsB s new wave of Follow Through appxoaches, 
the design of both the jpilot mad its cvolustion 
should b# grounded in reality. 

Jasm L. David 
From the coittlusion of 
"Making Evaluations of Follow Through 
Useful to Decision Makers" 



A. WHAT WE <lvE lEARyED ABOUT SCHOOL CHANGE 

•The Single school is the largest* ^mU of change,** as- 
serted Stuart Rankin, assistant superintendent of the Detroit 
Public Schools. "If more school districts would recognize 
this, more programs would succeed. The underlying reason is 
probably linked to the concept of ownersh^.!* . . 
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Energized by locally developed tests, Rankin claimed the 
best features of Dftroits* "measurement -driven" instructional 
program could easily be adopted and adapted by other districts 
and be as effective at lower grade levels as it is on the high 
school level « ' 

Drawing from experience, he affirmed that? "National eval- 
uations of programs are less important than evaluations 
designed and conducted locally f6r the purpose of improving 
ibcal programs," a point that was reiterated during the two. . 
day conference of evaluates and school administrators in 
Austin. I 

Commitment and collaboration are as important at adminis- 
trative levels as in the relationship of teachers to students. 
For that-reason, said Rankin, "if the district will make no 
financial commitment, then there is no coiranit^nt — just as if 
the teacher isn't convinced that tHe disadvantaged child^can 
learn, thi^n--the classroom program will not be a success." 

Wl^t would a school .look like, in terms of its specific 
policies and practices, if it fully erobodied these beliefs and 
commitments, and were informed throughout by a "measurement- 
driven program"? Such a school, Rankin suggested, would have 
the following characteristics: 

• Strong instructional leadership provided by the 
principal 

• Schoolwide emphasis on basic skills. 

• Clearly defined school objectives that are known by 
staff, students and parents 

• Carefully monitored student progress and regular feed- 
back to students 

♦ 

• High teacher expectations which, vividly communicated 
to students, stimulate peak performance by both 
teachers and students 

• Little or no ability grouping " 

• A* positive learning climate in school and parental " 
support of child-as-student out o^ school 

Localism as a key to success was underscored by Walter 
Hodges of the Parent Support Diagnostic Model developed by 
Georgia State University. Schools that opted to go into the 
"marriage" voluntarily were more successful than schools that 
•%*ere selected to participate in an experiment; on-site 
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AUSTIN— NIE CbNFEREKCE 
Febniary 19-20, 1981 



SESSIOK I ; What liave ve learned about iopleoentii^ and sustaining 
February 19 school 'innovation in early prograics the disadvantaged? 



9:30-11:30 



SESSIOK II : 
February 19 
1:00-2:30 



Papers: Walter Hodges 

., Stuart llankin '' * 

Discussants: Toxb Krueck 

linda Ballatas 

Vhat have ve iBBtneSi^fTom working vith schopls and programs 
that should be considered in future progtaai evaluations? 

r 

Papers: Eva Baker 
V Michael Fullen 



SESSION III : 
February 19 
3:00-5:00 



Discussants: Isaura Sant^ago-^Santiago 

Villiam Kutherfiord 

Robert McClure 

* ■ ^ , ' 

Vhat should one consider in the design methodology of an 
evaluation in light of the multiple alliances and in light 
of policy^ Implications? 

Papers: Mary Kennedy 

Jane B^via ^ 
« Tom Cook 



Discussants: 



Henrietta Xnapp* 
C, Lavor Lyio 
C* Thomas Fox, Jr. 
Ann Lieberman 



SESSION IV : 
February 20 
9:30-11:30 



Vhat haye ve learned frcns past evaluation and program exper- 
iences #ith regard* to instrumentation in latge scale progratc 
evaluation: 

Papers: Robert St. Pierre 
Richard Jaeger 

Discussants: Mi^g^ei Vang 

Gary Bo^ch ^ 
David ^Ibs . \ . 

^ \ ■ " > ;■ 

StSSIOK V : Iforking Grou^ Sessions 

Febmary 20 

l:00-4s30 A 
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sponsors did better than those who had to travel. The Bmaller 
the unit, the mbre effective and long-lasting the cljange, in- 
dicated Hodges on the basis of an extensive review of the lit- 
erature. 

Experience and research also deinons^irated that the 
smaller the unit, the more amenable it was to change. The 
child/parent learning environment was found to be roost respon- 
sive, the, classroom less so. The whole school was still more-, 
difficult to change and system^wide change was roost difficult. 

• Among his excellent suggestions for second generation 
studies, ohe which seemed especially timely, was a study of 
what happens when sponsorship is removed /to. determine how that 
process can be eased. "It's a dirty trick to reach someone tt> 
walk with crutches," he concluded, "when after they havfe f 
learned you intend^o yank back the crutches." 

> f ' 

B. UNDERSTANDING HOW AND WHY SpHOOLS CHANGE— OR DON'T 

Research and evaluation studies of successful programs are 
of limited Use, unless some school system chooses to adapt or 
adopt such a program. The first problem is disseminating the 
findings in such a way that provokes interest, the second is 
assisting in its adoption. 

'"How do we provide support, and information, to improve 
the effectiveness of education and maintain respect for the 
unique character of individual icKools and their needs?" was 
the question posed by Eva Baker of the UCLA Center for the 
Study of Evaluation at Los Angeles. "No educational policy 
will make much difference in students' performance unless it 
connects to classroom reality," she answered. 

* * • 

"Let's start from our desired goal," said Baker. "We 
want students who not only are competent but who are good at 
a whole range of intellectual endeavors. We believe we can 
get such students by high-quality, high- standards instruc- 
tion, and we have enough examples to know that it is possible. 
We also want schools to do what they can do best: teach and 
to share responsibility with parents and other institutions. 
We want to credit teachers for what they do or are ready to 
do with students. One powerful option is to provide informa- 
tion back to people, with some additional analysis of^tech- 
nical assistance,, to help them improve their efforts." 

iff 

Evaluators can help schools make changes they want to 
makfe by holding a mirror up to the schools so they can step 



THE PROCESS OF IMPLEMEKTATION 
♦ (Michael Ttallen, Ontario Institute for Studies in Education) 



THE CHANGE 



A new 
program 
or 

model , 



FACTORS AFFECTING 
IKPLtMENTATION 



K, Characteristics of the 
Change 

1. Need ' , 

2. Clarity /Complexity 

3. Materials Quality 

B. Implementation 

4. Adoption 

^. Staff development 

6. Time-line 

7. Internal /External 

C. District Factors 

8. History 

9. Administration 

10. Parents/Coemiunity 

D. School Factors . 

11. Principal 
. 12. Teachers 

■ . . ■ 

E. Extraneous Factors 
13. Unanticipated 

event s _______ 



IMPLEMEKIATION 



Changes in; 



1. 

2. 
3. 
4. 



Materials 
Structure 
Teaching 
Beliefs 



(involving 
teachers, aides, 
and pumts in 

instruction) ^? 



.OUTCOMES 



Achieve- 
ment At- 
titudes 



"XcplementatioD means changing practice, and its co^lexity is suggested 
by the diagram above. The mind vill be excused for boggling at the prob- 
lem, since it includes all of the above Issues and more; measuring all_^e 
inputs, measuring the various aspects of implementation, testing for avkri- 
ety of outcomes, interrelating all three sets in order to compare very dif- 
ferent FT models and %oi.ng beyond that to compare then with non-FT class- 
rooms. My own approach would be . . . (1) to develop camon measures of the 
inputs, (2) to explore some cpmmon ImploBentstion meaaurcs, but also rely on 
custom measare* of implementation unique to each model and (3) to use some 
comon outcome measures, but also rely on some custiM measures unique to 
itach model which will contribute to broadening the range of outcomes meas- 
ured."* 
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back and study themselves. "Right' now," claims Baker, "the 
evaluators* mirror comes from the amusement-park fun-housa and 
it is pointed at only one ann. The reflection is limited Vnd 
distorted." 

But we are not stuck with the distortion. We can set the 
mirror up to reflect local priorities. We can amass informa- 
tion that teachers can use to improve instructi'^n. We can 
find out if the school works well.. < 

To be "realistic," she cautioned, "we mttsj: not seek a 
single set of instruments which applies to everyone. "Rather, 
measures should be selected that conform as closely as possi- 
ble to the physical and psychological features of lthe school, 
emphasizing what the local agencies want," she concluded, 
echoing the principle affirmed by Stuajft Rankin. 



C. WHAT SHALL BE EVALUATED? AND HOW? 

What kinds of evaluation of new Follow Through approaches , 
would provide local decision-makers with what they need for 
intelligent adoption and adaption? Primarily, they' need to 
know what is likely to work in their own situations. To pro- 
vide just this information, a "multiple case study" approach 
which "can generate useful information to school, people, " was 
proposed by Jane David of the Bay Area Research Group. 

"Such case studies bXBHa"*"*He structure and generalizabil- 
ity of the experimental paradigm, with the richness and use- 
fulness of information gathered through the »ore meaningful 
case study." From them, one can draw valid conclusions that 
matter ; "x is true in large districts," or "y is true in 
schools with strong principals." 

i 

t 

Basically, the method of conducting such multiple site, 
structured case studies begins with "mapping" the "treatment" 
to be investigated and the context in which it will be ob- 
served. After a small selective sample of varied sites is 
chosen, field researchers are provided with an interview guide 
or topic outline to guide their observations. Each data col- 
lector returns from the field with siimilar types of informa- 
tion gathered on the same topics, yet these data also reflect 
the unique characteristics of each situation. 

The data is then analyzed rigorously, to make c6mparisons 
in which tentative conclusions based on one case are system- 
atically tested against each of the other cases. If they do 
not hold up in their original form, the conclusions are modi- 
fied so that they do; if the amount of modification required 
to make the conclusions hold in all instances is excessive. 
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they are dropped as ungeneral liable site-dependent phenomena. 
"The conclusions that remain after this obstacle course of 
pairwise coroparisons are finally presented with illustrations 
drawn *rom the cases," according to David. "Overall, the 
multiple case study approach maxintixes the likelihood that the 
evaluation will be both relevant and meaningful to those ^ 
closest to the difficult task of bringing about school change. 

The research priorities suggested by ti<e NIE in their 
plans were too narrow in two respects, argtied Thomas 'Otjox of 
Kbrthwestern University. First, they would use panel measures 
only during the time the childrfen vrere in Follow Through, and 
-second they would measure ortly implementation variables. 

Cook argued that the "panel it udy method,.- described in 
his paper, could be used to answel^ more important questions 
with no undue cost increases: questions such as, "How effective 
is the program, what kind of an im|>act is it making on students' 
and, "What elemtnts pi its implementation are responsible for 
the results that are being observed?" 

He points out that misguided research, even if successful, 
may document the implementation of services that are of little 
utility, i.e., children spending more time on ineffectual 
tasks.. . 

"If ever there is or was a time when school districts 
were not interested in trying new innovative approaches to 
education, it will be the decade of the eighties," said Mary 
Kennedy of the Huron Institute. She cited the severe decline 
in funding, and enrollments, and the primitive 6tate of the 
art of evaluation (which to date has devised f^w acceptable 
instruments)^ Therefore,, the best use of the HIE research 
funds, according to Kennedy, would be to^^ncrcaseour under-^^ 
standing of -disadvantage," the process of~lRTtew«Bn, the 
process of implementation, the social settings in which inno- 
vations meet disadvantaged children, the effects of federal 
regulations, and the nature of incentives for change in school 
systcsms. ' ' 

For example, a study into the nature and implications of 
disadvantage might use intensive case studies of individual 
families in Individual Follow Through coaanunities to learn how 
schools, and Follow Through in particular, are perceived by 
poor families. Specifically, how a program that offers com- 
prehensive services to poor children, and attempts to make 
their parents more self-sufficient, interacts with and influ- 
ences the poor. 



Studies of how to Improve the administration of services 
might focus 0 the nature of inter-agency agreements that are 
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necessary to coordinate services, pr the best way to reduce 
delivery cbsts without impairing the quality of services. 

These suggestions were offered by Kennedy as a "way of 
emphasizing that there are a great many important educational 
questions that can be answered that do not involve the testi*ng 
of educational models, and they are questions that, can be ad- 
dressed through a f ederally-coctfrdinated research program." 



D. CAN WE BU-ILD ON THE DATA WE HAVE ALREADY COLLECTED ? 

"No single standardized achievement test should be used 
for overall evaluation of the Follow Through program," 
asserted Richard Jaeger of the University of North Carolina 
after reviewing the testing literature in the field. Such 
tests are not valid for assessing the impact of the diverse 
curricular approaches represented in Follow Through, he ar- 
gued. Moreover, they differ markedly in their detailed con- 
tent and in their congruence with the content of the basic 
skills curriculum materials widely used in the schools. An 
extensive review of the literature by Jaeger revealed that 
while thdre was never more than a 60 percent overlap betse(een 
the test items and the curriculxim materials in some cases> 
there was no overlap at all in other cases. Finally, he 
pointed out, ther^ is a "tendency to ovcrintcrpret standard- 
ized test results wheh judging the merits of compensatory ec^- 
cation programs. " 

Of course, the major content validity problems could be 
avoided by using, a number of properly-equated standardized 
achievement tests for evaluation of the Follow Through pro- 
gram, rather ^han using just one test. The advisability of 
conducting such a major equating of tests was examined thor- 
oughly in Jaeger's background paper. Reviewing the major ETS 
Anchor Test Study, designed in 1971 to improve national eval- 
uation of ESEA Title I programs, the author noted that al^ 
though this effort did not prove to be useful for ESEA evalu- 
ation (because of a change in the federal approach to Title I 
Research) it clearly established the feasibility of equating 
the reading comprehension and vocabulary subtests of different 
test batteries, eveji though they were not designed to be psy- 
chometrically parallel." Jaeger asserted that a study equat- 
ing corresponding subtests of test batteries could yield "sub- 
stantial benefits" if the u6e of standardized testing is still 
pervasive and if relatively few tests are widely used in the 
eaxly elementary grades. 

Everyone agrees there is more tq be learned from Follow 
Through. The question Robert St. Pierre of Abt Associates 
asks is, "What research is most cost effective?" 
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Tifo areas of future research' were reconsnended by St. 
Pierre: (1) the potential delaye<^ effects of Follow ThrdXigh, 
and C2) research on sponsorship as a mechanism for change in 
schooling. 

• Potential delayed effects'* commends itself for jBtudy for 
several reasonSf* the author contended: 

• Since a better chance at life success is the ultimate 
goal of the Follow Through program, it should be in- 
vestigated 

• • Follow Through sponsors who predict salutary long-term 
effects favor such a study 

* * ■ . 

• The data to perform delayed effects £xist in a reason- 
able form ^ 

• Studies of post-Follow Through effects would be rela-r 
tively inexpensive f and could provide infcmnation in a 
timely manner. ' 

,v Studying sponsorship could lead to generalixable strate- 
gies for educational change, according to St. Pierre. "Spon- 
sorship is perhaps Follow- Through *s roo^t creative contribution 
to educational change. • Yet Follow Through has been, so focused 
on ehild outcomes that very little effort has been devoted to 
understanding the key Qoncept of sponsorship, defihing.it, or 
comparing it with other mechanisms for change," he asserted^ 

f, r 

1. miat S hall Be Measured? How Shall We Measure It ? 

All ney programs should have tests and data that ^re ' 
tailored to the site yet which ^an be compared across sites, 
indicated the work-group on *Iri"strumentation: Issues Related 
to Measuring '|*rogram Effectiveness." ^ 

The choice of instruments can be' eclectic — (surveying, * 
synthesizing and traditional evaluation instruments can all be 
usefii^) if they serve "^o^take the temperature of the school." 
Because a program's success is many-faceted, it should be mon- 
itored on many levels. 

TO monitor students: 6pnsider achievement tests*' (non-paper 
and pencil as well as the more traditional fomis) ; self- 
concept scales, and tests that measure students' atti- 
tudes towards school and learning. (The te^ts should 
Bessure development over time . ) . . 
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TO loonitor teachers: consider instructional practices, roan- 
^ "^Sg^ent skills, attitudes such as ^^P^f ^^^^^^^^^ H^S. 

ness to change, academic Standards, relations with stu 

dents and other school relationships. 

To monitor principals; consider management skills, informa- 
tion actjuisition, leadership. * ) 

TO monitor school environment: ^olledt climate data such as/* 
safety, violence, /referrals, substitute attitudes, 
teacher-absenteeism, program fcoordination. 

^ TO xnonitor classrooms: 'consider tracking-data, ev^^^ of^ 
planning, the resources availablfe and the resources usea. 

To monitor parents: consider their stated satisfaction with ; 
the quali.ty of the school and t^eir children's Perform- 
ance? their awareness of problems, their involvement in 
school ^tivities and in school decision making. 

*.The croup rejected the. notion that an- appropriate common 
ins^rS^h^o? fudging quality and standards 
tie* exists. But they did suggest thlt With NIE s jeXp, 
schoordistricts could enlis^ technidfl assistance to adapt, or 
develop measures that would meet locrfl needs. 

2. Can the Study of One System H elp Others? . • 

Based on the assumption. -that a detailed,, thorough study 

of one Itst^cfs atte^^^t to i^Pl^-^^t c^^^Sons: ""^Sues r"' 
4^ho i^^kehoD on "Local Evaluation Considerations: issues kc 
SteTto S^sigSing an Evaluation Responsive to Local Formative 
iSdiJ^ative Seeds,- made the following, recommendations: 

1 Distticts should be studied in depth. ^Rather than 
. finding many districts mbdestly, should select 

^ tSo districts and fund them at one million dollars 
each- to do a thorough job of evaluating their capa- 
city to implLent, aid the process of implementation.. 

2 The program to be implemented should «3dress whole 
' schools rather than selected classrooirfs. 

3^ Each district chosen should have a demonstfatcd 

record of successful evaluation and research efforts 
as well as a strong longitudinal data base. 

A The atudv should adequately measure the "cbool^s 
^Eco^oS^ Attitudes, ^chSol dimate, etc should 
so well documented that others will' *>e^*ble to use 
the informalion for purposes of transportability.^ 
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5. the study shoulg be fund^ to last at least five 
years. , • 

• 

3. Would A Multi-site Study Be Prefera\)le ? V 

A responsible study should have more than one site per 
model, each site should have one or more schools, and, each 
schbol should invorvfr-aj^ K-3 x:lasSrooms, advised the group 
discussing "Cross-Site C^t^iderations: Issues Related to , 
Designing An Evaluation Usei\il. for Program Adopt lon/^aptioti 
D^cisiojis^." ^ * 

Sites should be heterogeneous. Ideally, this project 
would examine three distinct models with three sites apiece, 
which differ from each other in ethnic mix, size and location 
(rural, urban) &hich will help evaluators determine the trans- 
portability of the models. 

Local districts could, be involved in developing Follow 
Through models in one of two ways: a ^single si*€ could spon- 
sor the model as has . been ^ne in the past by F6now Through 
,^ self -sponsored *ites, or ^ group of sites could form a con- 
s;^^ium to sponsor a modjpl* ^ - 

• i dvestioning the advisability of NIE supporting local 
eva>vStion^ the group pointed out that local evaluations are 
oftendone poorly except for large districts wx?A specialized 
% staffs, and^ven when done well, they are mainly useful for 
-the local district. 

' If threi* models are funded, the group would! suggest allo- 
, eating the $2.5 million as follows: U- \ 

$500,000 for each of the three models 

$500,000 for an external evaluation 

$500,000 for other NIE research, evaluation, 
analysis, etc. 

A« two-pronged effort that would capitalize on what cur- 
rent Follow Through sponsors have learned was proposed by the 
group. By Evaluating a number of existing Follow Through- 
lites wheie sponsors have had several years to develop their 
models and have strong working relationships with Pf^" 
Bonnel, the study would give compensatory education J§f ^e^^ 
possible chance to fulfill and demonstrate its potenti*^ 

In order to/4ddress NiE's desire to fund low-cost, trans 
portable programs, it was proposed tKat the three-sponsor. 
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three-site research program described above include an' element 
of sponsor withdrawal-' The evaluation would follow the start- 
up, th^ implementation as it proceeds and the process of spon- 
sor withdrawal after two years of the program thereby docu- 
menting the relationship between implementation and outcomes 
throughout the process. 

4. Getting the (Documented) Word Out 

A program succeeds in District A, What can be done to 
get District B to consider it? 

That was the focus of the work-group on "Communication/ 
Documentation: Issues Related to Increasing the Utility of 
Evaluation Docismentation. " ^ 

Inducements in the form of grant support is one method of 
getting new programs into the schools. It induces schools to 
try new programs which might succeed sufficiently to be ab- 
sorbed iAto the system.^ But the problen with this propedur^, 
noted Eva Baker, is that although schools and, systems may jump 
through hoops to get money ,^ such programs are often regarded 
"more as an obstacle than as an integral and useful practice 
for the school." People may comply with the, required proce- 
dures mainly to get "kidney-shaped tables and chalk," but if , 
the program* is viewed as a means to ano€her end, 'it probably 
will not contribute to enduring local reform. 

Communication may be the answer — if each subgroup in the 
audience of decision-makers is^ treated separately and ti>e ap-' 
proach to each is individualized, the group advised.- For exam- 
ples 

m When ad^essing the general public, focus on outcomes 
and use simple percentages rather than complex statis- 
tics. Via^B it as clear as a Reader's Digest piece. 

9i When addressing the educators, parents and community 
decision-makers, be specific about student outcomes, 
cost-effectiveness, and how the "target" populations 
will be selected. Tell them what teachers will be ex- 
pected to do differently, demonstrate the programs' 
operation and materials and, above all, pitch the talk 

' at the language level of the group. Avoid jargon. 

• When addressing district administrators, be specific 
atiout expected outcMsbs and present in detail the ad- 
I, ditional jrequirements or impositions of the program. 
Demonstrate the ^materials and present the research 
^sults in Nontechnical visual way. 
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• When aadressing State DejJartments of Education be • 
specific about how the priograjn will or will not be in 
compliance with state regulatipns. Tell them what 
kinds of dissemination the district will engage ink 
arid, if possible, get a district superintendent to 
thi presenting. 

/ • When addressing federal administrators be specific 
about the cost-effectiveness and how the program is 
< to be monitored at the site level. 

A final bit of advice from the group: When possible, 
have an administrator address administrators, a teacher ad- 
dress teachers, etc. 

A program is being* implemented in District A. What can 
be done to get the staff to cooperate enthusiastically?. 

Communica tion for Improving Practices can be iinproved by 
focusinAon its key function: ^to help practitioners know xf 
what they are doing is consistent with what they are supposed 
to be doing, and that theit work is having an effect on the 
students. Therefore, good communication is frequent, ti®eiy 
and appropriate. 

Tests should help teachers monitor themselves W compar- 
inq test results with objectives. ^School people want to fxx 
things day to day when they go vrong. The evaluators and test- 
ers should be on locatipn, and the test results should be 
available to practitioners within a day or two of the assss- 
ment. * 
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Rarelv in the history, of educational research funded by ' 
the fSSeral government., hL a new program beren pl^ed in such 
Sos^con^ltation with the field and with the Pf'^i^^lnhia 
Interest Among the participants at Portland, P»^il«<»«iPj;i* 
iSd iSstin.^d among the national experts who prepared back- ^ 

?S?e£o? Soliciting such input and . J^f^^f^'j "JJ^^t!^!SSca- 
Jears of aonitdring and appraising fl^T 
tional research^- Vaid one veteran observer «>f the tield, 
-thi^ il the mo^t lincere ef tort I've been or heard of, to 
r^^llv listen^o what theorists and practitioners have to say, 
^ bSild our convictions into future funding pribriti^s." 

' The oreceding pages have endeavored to give a readable 
•ni^ fSthful SccSSnt of the significant things that were said 
if efch SrtbriSn?erences. Inking back over thte entire ser- 
?!- - themes call for concluding emphasis. As might be 
les, a J^^'^iJ^ Mre not radical or astonishing. Rather, they 
Sr^SvicSSrs ^hlcrseSled^opervade much of the discussion. 

I Th. focus throughout on the better wanagwnt of early 
<U,iwS^ ISpe^.toly education pro^^ 

widi^epruid «»«r'^«H''"/l!^Lch lf^ta4 S'SS*!". doni well . 

particular pedagogical "PP";™' " ' ' 

. the. aystem will work to benefit children. 



40 



37 



But how dbes one get a systean ^o work well? It would be 
a relief to^find a |>erfect program and itnpXement it wholesale, 
but no such solution is feasible. Any educational system 
deals with iiuman beings who bring their personal ideologies , 
their life experiences, and their own ^teaching styles to. their 
w>rk^ No matter what their training, they bring themselves 
Into the classroom, and they interact with children who also 
have distinctive life experiences, learning styles, aftd his- 
toriies* 

The one predictable el^nent of implementation is that 
researchers and developers miist be prepared to work with these 
variables^' They must learn frwn past . experience that no sin- 
gle model can satisfy all intierested parties. Moreover, un- 
planned . disruptions are inevitable: children relocate, 
teachers leave, and school systems close for lack of funds, 
snow days and strikes. Clean scientific research falls victim 
to these messy realities.' 

Despite iihis, members of the Follow Through community can 
offer helpful advice to contemporary education programs of the 
futi^re. For example, experience shows that interested parties 
will be more committed: 

• i£ they are consulted. ' . 

• if they have the solid support of their peers 

• if the program solves, problems they themselves have 
found troublesome 

• if they fully understand the point of the program and 
have a stake in its success 

• if the program does not violate what they "know* about 
the wrld. 

Perhaps the roost important point that "went without say- 
ing" was the crucial— one might even say crushing— role of 
poverty. The subject was not insisted upon in the discussions 
or papers; yet it was a clear .undercurrent in many of them. 
For example, parent's renewed sense of self respect was 
widely regarded as essential to encourage better learning in 
their children. Out-of-session conversations preoccupied with 
the loss of GETA funds and the coming cutbacks in health and 
social services to the struggling poor, clearly revealed the 
relevance of such economic and social conditions on the educa- 
tors' success with disadvantaged youngsters. 

When hopelessness surrounds children at hone and on the 
streets, they su/fer ,as learners. Resignation to an unfulfilling 
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future of dininiBhed lif^^opportunities saps motivation. 
Neither new classroom techniques not; better sianag^ent can 
overcome ninds afflicted with despair at an early age. 
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